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Abstract. We study the belief propagation algorithm for the graph bi- 
partitioning problem, i.e. the ground state of the ferromagnetic Ising model at a 
fixed magnetization. Application of a message passing scheme to a model with a 
fixed global parameter is not banal and we show that the magnetization can in 
fact be fixed in a local way within the belief propagation equations. Our method 
provides the full phase diagram of the bi-partitioning problem on random graphs, 
as well as an efficient heuristic solver that we anticipate to be useful in a wide 
range of application of the partitioning problem. 
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1. Introduction 

Graph partitioning problem was one of the first optimization problems treated 
with methods of statistical mechanics of disordered systems [TJ |2]. Since then 
other applications of the theory of spin glasses in hard optimization and constraint 
satisfaction problems attracted a lot of interest and many remarkable results were 
obtained. As anticipated in the early works [2 [3], understanding of the energy 
landscape and the phase transitions in the space of solutions leads to understanding 
of algorithmic hardness of the problems [H [5], and even more remarkably it leads 
to a development of a new class of heuristic algorithmic techniques 0]. Nowadays, 
the cavity method [5] serves as a state of art technique for understanding random 
optimization problems, and its application on a given instances of the problem is a 
base for a class of one of the most promising heuristic solvers, known as message 
passing algorithms in computer science. 

Despite all this activity in the field, nor the phase diagram neither a message 
passing algorithm for partitioning a graph into two groups of a given size has been 
worked out. The main reason that makes the graph partitioning a tricky problem 
to treat is the existence of a global constraint that fixes the size of the two groups. 
The aim of this article is to fill this gap, and give the phase diagram of the graph 
bi-partitioning on sparse random graphs and associated belief propagation algorithm. 
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1.1. Partitioning problem: Setting and applications 

A graph G(V, E) is given by the set of vertices V and edges E. If an element (i, j) 
belongs to the set of edges we say that vertices i and j are connected. The graph 
bi-partitioning problem consists of dividing vertices of the graph into two disjoint sets 
of a given size, so as to minimize the number of connections between vertices from 
different groups. The problem is known to be NP-complete [7], and hence there is a 
good reason to believe that no exact polynomial algorithm exists. 

The graph partitioning problem is encountered in many important applications. 
To give few examples: In an electric circuit design one needs to know on which board 
to place the different components to minimize the number of links between different 
boards [S]. In parallel computing one has to partition data and tasks among several 
processors in order to minimize the communication between them [S], Partitioning is 
also closely related to data clustering and community detection [TU]. The list could 
continue for long, and it is hence crucial to develop efficient heuristic algorithms that 
give good solutions to the problem. 

A large volume of literature on heuristic methods for graph partitioning exists. 
One of the early fundamental works in the field is [llj . its running time is, however, 
0(N 2 ) so it is no longer used in practice. Simulated annealing techniques can be used, 
see e.g. [T^l H3]- A local search based methods such as the extremal optimization of 
[14] were suggested. There is a whole class of spectral partitioning methods that use 
the eigenvectors of the Laplacian of the connectivity graph, see e.g. [15] . However, the 
current state of art method for partitioning, that is used in most practical applications, 
is based on the multi-level programming: The nodes are grouped into super-nodes 
and the super-nodes grouped again, at the end the system size is very small and the 
problem is solved exactly and the grouping of nodes is then unwrapped. The multi- 
level programs use elements from many other approaches, see [16] for an excellent 
review. 

We do not anticipate that belief propagation developed in this paper, will 
be by itself competitive with the highly tuned implementations of the multi-level 
methods. However, we do anticipate that it can be used as a component of these 
implementations. For example, in the multi-level algorithms one needs to estimate 
the probability that two nodes can be grouped in the same super-node — this is 
exactly what belief propagation is designed to compute very fast and efficiently. 

The graph bi-partitioning problem is equivalent to finding the ground state of the 
Ising model with fixed magnetization. The energy in the Ising model is given by the 
following Hamiltonian: 



(ij)eE 

where Si is the Ising spin (either +1 of —1) on the i-th vertex of the graph. The 
magnetization m, — 1 < m < 1, is given by 



where N is the number of vertices. Therefore, the problem of finding a configuration 
of spins that minimizes ^ while demanding magnetization m to be fixed is equivalent 
to dividing vertices into two groups of size N(l + m)/2 and N(l — m)/2 such that 
the number of links between them is minimal. For m — 0, the graph is divided into 
two groups of equal size, i.e. the graph bisection. The cost of a graph partitioning 




(1) 




(2) 
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at a given magnetization, that we call b(m), is given as the number of edges between 
different groups divided by the total number of vertices. The relation between b(m) 
and ground state energy E(m) of the Ising model at magnetization fixed to m is 

E(m) + M 

Km) = ' ( ) 

where N is the number of nodes, and M the number of edges. 



1.2. Previous results on hi- partitioning random graphs 

In graph theory estimating the asymptotic size of the bisection width in random 
regular graphs, i.e. graphs of a fixed degree chosen uniformly at random from all the 
possible ones, is a classical question. Many upper and lower bounds were derived. The 
currently best known upper and lower bounds on bisection width in random regular 
graphs are by [TTl [TBI H3 EQl ET] and we summarize their numerical values in Table Q] 
and Fig. HI 

For Erdos-Renyi random graphs with N — ► oo vertices and mean degree a (every 
edge is present with probability a/(N — 1)), the size of the largest component is 
gN + o(N), where g satisfies the following equation: 

g = l-e- a ". (4) 

In order to divide the graph into two parts of size N(l + m)/2 and N(l — m/2) such 
that the number of edges between the two is zero, the size of the largest component g 
must be at maximum (1 + m)/2. That is possible for average degree a < a s where 

2 , 1 -m 

a s = -— log——. 5 

1 + m 2 

For a > a s an extensive number of edges needs to be cut in the minimal bipartition. 
The value a s is hence in a sense the satisfiability threshold for graph partitioning of 
Erdos-Renyi random graphs. This is further discussed in E2], where the authors also 
obtain an interesting upper bound on the bisection width (m = 0). 

In statistical physics many articles addressed the random graph bi-partitioning 
problem, see e.g. El QH ES El E3 El EH HFJ , but as f ar as we can tell they address 
only cases where (A) the magnetization is fixed to zero, (B) the fluctuations in the 
degree of the random graph are negligible, i.e. the graphs are either dense or regular. 
The computational techniques used in the above mentioned papers do not generalize 
to the non-zero magnetization case nor to graphs with fluctuating degree, as e.g. to 
the Erdos-Renyi random graphs. We will give a more detailed explanation of why the 
techniques do not generalize in section l4~3l This also justifies novelty of the approach 
developed in this article. 



1.3. Contribution of this article 

If the ground state energy of the ferromagnetic Ising model (fTJ) was a convex function of 
the magnetization m then an external magnetic field (playing the role of the chemical 
potential from the grand-canonical ensemble) could be used to compute E(m) with a 
standard cavity method |29j . However, random graphs are mean field topologies and 
the energy at fixed magnetization E(m) does not have to be and in this case is not a 
convex function, similarly as in the fully connected Curie- Weiss model. The problem 
of imposing the value of the magnetization is hence more challenging. 
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A method to explore the non-convex parts of thermodynamical potentials within 
the Bethe-Peierls (Belief Propagation) approximation was suggested in [30] , and used 
later e.g. in [311132] . The main idea is to introduce an uniform external magnetic field 
(or chemical potential) and adjust its value after every update of the local cavity fields. 
We use this method for partitioning graphs, and we argue that it (or its generalization 
to the replica symmetry breaking scheme) is asymptotically exact on sparse random 
graphs. 

The main practical contribution of this article is the belief propagation algorithm 
for graph partitioning problem that we believe to be of use in the various applications 
of the problem. We study the behavior and performance of the algorithm on random 
graphs but we anticipate it will be meaningful and useful for other families of graphs, 
complex networks for example. 

We also compute the phase diagram of (p} at fixed magnetization. In [22] it was 
argued that in the Erdos-Renyi graphs at zero magnetization the glassy transition 
happens at some average degree strictly larger than the satisfiability threshold, 
a c > a s , we indeed confirm this conjecture, we compute a c and several other quantities 
of interest. 

An interesting side remark, discussed in section 14.31 concerns the case treated 
in the previous works: the regular random graphs at zero magnetization. There the 
average properties of the graph bi-partitioning are equivalent to those of the spin glass 
problem. We argue why this equivalence does not generalize to non-zero magnetization 
or non-regular graphs. More detailed discussion about the equivalence can be found 
in 1331 . 



2. Cavity method at fixed magnetization 

As we explained in the introduction, the graph partitioning is equivalent to the 
ferromagnetic Ising model at fixed magnetization m*. The magnetization will be 
fixed via an external magnetic field h which appears in the Hamiltonian as 

H h = - ]T SiSj-hJ^Si- (6) 

(ij)eE 



The ground state energy density of fl} and ([SJ are related via the Legendre 
transformation e(h) = e(m) — hm, so that the parameter h has to be chosen such 
that 



dh 



= -to* . (7) 

h* 

If e(h) is the ground energy density of ([6]) with field h corresponding to magnetization 
m, the corresponding partition cost ([3]) of the graph is 



e(h) + hm + § 

= 2 ' 

where a is the mean degree of the graph. 



(8) 



2.1. Belief propagation equations 

The Bethe-Peierls approximation, or the Belief-Propagation equations, aims to 
describe the Boltzmann measure of ([6]) 

-f}H h {{Si}) 

tiiSi}) = s . ( 9 ) 
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where is the inverse temperature. The graph partitioning problem corresponds to 
— > oo. In this section we summarize the well known belief propagation equations 
for this problem. For a detailed derivation see [3"4l, 155] , 

In the most standard form of belief propagation equations [34] one introduces 
ip l ^* J to be the probability that variable i takes value Sj given the interaction on (ij) 
is absent. On a tree (cycle free) graph then 

^■ = _i_ e ^ n ($>^r)> do) 

k£di\j Sk 

where Z l ~ >J is normalization ensuring ip+i' 1 ^r^-i = !• After a fixed point of eqs. (| 10(1 
is found the Bethe free energy (or the log-partition function) is given as [M] 

- (3F(h) =l 0g Z = J2 lo § zl - E lo S Zij > ( n ) 

where 

* = £y** n (E e/3SiSfc ^r)' ( 12 ) 

Si kedi s k 

Z ij = Y,^ S ^ l s7^sT- (13) 

Si,Sj 

At a given value of the external magnetic field h the average magnetization is computed 
as m = — [dF(h)/dh]/N, using (fTT|) one gets 

m = — > ^- . (14) 

In order to write the zero temperature limit, (3 — > oo, of the above equations we 
introduce more suitable messages (usually called cavity fields) h % ~*^ 

e 2fih'-> = f±L (15) 

One then obtains equations equivalent to the replica symmetric equations in [29] . The 
self-consistent equations for messages (fT0|) become 

h l ^=h+ J2 [max(l + /i fe ^,0) -max^.l)] = F({h k ^ 1 }) . (16) 

kedi\j 

Note that the term in the sum is —1 for h k ^ 1 < — 1, +1 for h k ^ % > 1, and /i fe ^ 4 for 
— 1 < /i fc ^ 4 < 1. The Bethe estimate of the ground state energy is 

E(h) = j2 Ei -J2 Eij > (1 7 ) 

' (ij) 

where from (|12II13[) we obtain 

E i = /i + a + 2^max(0,/i fe ^ ?; ) 

- 2max[ft + ^max(l + /i^ ! ,0),^max(/i^,l)] (18) 

E 13 =1 + 2 max (0, h 1 ^) + 2 max (0, h j ^ 1 ) 

- 2 max (1 + + 1, h^ j ) . (19) 
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And finally the contribution to the sum in the expression for the magnetization 
(|14p is in the zero temperature limit equal to +1 if 

h+Y^ [max(l + /i fc ^O)-max(/i fc ^\l)] >0, (20) 

and —1 otherwise (if the two terms are equal the contribution is 0). For notation let 
us call this function 

m = M(h,{h i ^ j }). (21) 



2.2. Population dynamics at fixed magnetization 

In order to calculate the average ground state energy (|17j) . and thus the partitioning 
cost b, for a given ensemble of random graphs one implements the population dynamics 
method 0[35]. 

In the standard population dynamics one would update eqs. (j 1 6|) with a given 
value of external magnetic field h till convergence or till maximum number of iterations 
and then one would compute the ground energy and the corresponding magnetization. 
If this is done with the above equations for graph bi-partitioning then the resulting 
magnetization will always be either +1 for h > or — 1 for h < 0. We however want 
to find the ground state energy at magnetization values — 1 < to* < 1. In order to 
do that we will not keep the external field h constant. Instead after every iteration of 
(fT6|) we use the current valued of fields h 1 ^ and update the value of h in such a way 
that m* = A4(h ae w, {h 1 ^-*}) where to* is the desired value of the magnetization. The 
resulting population dynamics code is sketched in algorithm [TJ 

Note that the algorithm [1] uses bisection method in each iteration in order to fix 
the magnetization. For e?-regular graphs, M(h, as a function of h for given 

values of {h l ~ >J } is continuous monotonic function and therefore the algorithm always 
manages to fix the desired magnetization. For general graphs A4(h, {h 1 ^ 3 }) may have 
less well behaved form and we will discuss that in section [3l 



2.3. 1RSB at fixed magnetization 

As may be anticipated from the relation between graph bisection and the spin glass 
[2] the belief propagation equations (replica symmetric approach) are not always 
asymptotically exact for the graph bi-partitioning. Instead in some regions of 
parameters the problem is glassy and the replica symmetry breaking approach is 
needed for an exact solution, just like in the Sherrington-Kirkpatrick model 137j . 
The replica symmetry breaking approach for sparse random graphs was developed in 
[6l [29] and is well established. Hence we only point out the difference in the equations 
that leads to fixing a non-trivial value of the magnetization. 

In order to write the 1RSB equations we follow closely the approach of [29 . We 
introduce a complexity function S(-E), i.e. number of thermodynamical states at a 
given energy, and its Legendre transform Q(y) also called the replicated potential 

-y*(v) = -yE + J:(E) > (22) 

Every thermodynamical state has a corresponding value of the cavity field h l ~^ 3 and 
according to the self-consistent equation for the distribution of cavity fields over 
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Algorithm 1 Population dynamics algorithm for BP on d-regular random graphs 
with fixed magnetization m* 

Randomly initialize messages h(i), i — 1, 2 . . . N 
for j = 1 to max do 
for i = 1 to N do 

Randomly select d indices in k = 1 , 2 . . . N 
Calculate h(i) from {h(k)} using eq. (fl~6f 
end for 
hi<-h-l 
h 2 ^h + l 

while \h\ — h 2 \ < criterion do 

Calculate magnetization m with external field h using eq. 1211 
if m < m* then 

hi <— h 
end if 

if m > m* then 

h 2 <— h 
end if 

h «- {h x + h 2 )/2 
end while 
end for 

Calculate E using (|17p (averaged over some number of iterations) 
return E, h 



states is 

P*-J(fri-i) = _L_ j Y[ &P k ^{h k ^)e-y E ^ 3 5[h^ -T{{h k ^})], (23) 

kedi\j 

where T{{h k ^ 1 }) is defined by eq. The reweighting factor is defined by 

E^i = — lim^oo log where is the normalization constant in (jTUJ) and 

is given by an equation analogous to (fT5|) . Once a fixed point of (f2"3"]l is found the 
potential $(y) is computed as follows $(y) = $ l — J^. . with 

e"^' = / e-"^ , e-^*" = f e~ yE ^ , (24) 
Jpop Jpop 

where the notation J POP = fJlkedi\j dP k ^ z (h k ^ 1 ) and the energy contributions are 

given by (|18H19p . The energy of the system is then computed according to (|2"2"j) as 

E = E£ i -Ey£ ij with 

pi _ JPOP ± C pij _ JPOP 12 ' C (0 r\ 

JPOP^ JPOP v 

And the magnetization m — m l /N , where 

flfi f Ml -yE* f ML E i ~yE % 

™i _ uc _ JPOP dh e | „.JPOP dh ^ e 



dh f POP e-v* ' y " J POP r^ 

dE i e -yE i 



JPOP 1 ^ t JPOP dh - (9 fo 

v (fpopt- yE r 
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Note that = ±1 depending on the sign in eq. pTjl . 

Again the only difference between the usual fRSB and 1RSB at fixed 
magnetization is that after every iteration the external magnetic field is chosen a 
new value such that magnetization computed from (|26p is equal to the desired value 
to*. 

Solving the fRSB equations is often tedious and to obtain the phase diagram it 
is often sufficient to investigate the convergence of the belief propagation iterations. 
This is equivalent to analyzing the local stability of the replica symmetric solution 
towards replica symmetry breaking, as done originally by de Almeida and Thouless 
|38j . Within the population dynamics we use the noise propagation method (for a 
derivation see appendix C of [39]). In the population dynamics algorithm together 
with cavity fields h l ^° , one keeps track of the quantity 

k£di\j 

after every sweep of BP iteration we normalize the values v 1 ^ 3 by A in such a way 
that Y](v l ~ f:l /A) = f . Parameter A then plays a role of a certain Lyapunov exponent 
and the belief propagation does not converge if and only if on average A > f. We 
have found that BP never converges on regular graphs for any value of magnetization 
— f < to* < f . Nevertheless, the value of the energy calculated with BP gives a good 
lower bound on the actual energy of the model [3D] . For the Erdos-Renyi graphs with 
given magnetization, we found a phase transition from a replica symmetric region 
where BP is asymptotically exact to a glassy region where RSB solution would be 
need to obtain the asymptotically exact value of the ground state energy (this phase 
transition is shown in Fig. [1} . 

3. BP as a heuristic solver 

Equations for the belief propagation derived in the previous section can be used on a 
given graph as we sketch in algorithm [2] 

The parameter memory, which we set to 0.7 in our simulations, is introduced in 
order to prevent messages from oscillating. If the algorithm does not converge after a 
given maximum number of iterations, it is terminated. However, even if the algorithm 
does not converge, the calculated E still provides a reasonable estimate of the bisection 
cost that is on average a lower bound of the true average cost. 

In the presented algorithm, we introduced a slightly different method to fix the 
magnetization by manipulating h. In the algorithm^ we sort all the local cavity fields 
and set h so that iV(l — to)/2 of them are negative (or zero) and the rest positive 
(or zero). It follows from the definition of messages (fT5|) that the positive value of 
local cavity field means that spin on this node is more likely to be equal to 1 , negative 
means that the spin is more likely to be —1. If the local cavity field is exactly equal to 
zero, the spin in a given node is unbiased (free). This can be used to actually obtain a 
graph partition. However a decimation technique, algorithm [3] achieves much better 
results in particular when many free or almost free spin are present, reported in Figs. [3] 
andg] 

Note that in the present form the decimation solver has running time quadratic 
in the size of the system. However, linear running time can be achieved without 
significant loss of performance by decimating a finite fraction of spins after every 
iteration, as in the survey propagation algorithm of [4] . 
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Algorithm 2 BP algorithm for partitioning of a given graph 

Vi, j Initialize messages h l ~* 3 and field h randomly 

iter <- 

repeat 

for all i G V do 

convergence <— 

local_field[i] <- £ fceai [ max C 1 + ^%0) - max (/i fe ^, 1)] 

^new ^+ local_ficld(i) - 

convergence <— convergence + | h^J, — h l ^i | 
<- memory*/^ 1 + (1 - memory)*/^ 
end for 
sort (local-field) 

h = - locaLfield[iV(l-m)/2] 

iter <— iter +1 
until convergence < e OR iter > maximum iterations 
compute E using equation (|17p 
return E, h 



Algorithm 3 Decimation algorithm 
repeat 

Run algorithm [2] 

Choose a vertex i such that locaLfield(i) is the highest (or lowest if this is an even 
iteration) and fix all outgoing messages from this node to +oo ( - oo for an even 
iteration). Fix spin in vertex i to +1 (—1 in even iteration) 

until Number of fixed spins to +1 or —1 reaches the value required to fix desired 

magnetization m 



4. Behavior of the method and results 

In this section we discuss the behavior of the belief propagation algorithm of random 
regular and Erdos-Renyi random graphs. We, however, anticipate that qualitatively 
similar behavior as on the Erdos-Renyi random graphs will be seen on other graph 
families. 

Phase diagram of Erdos-Renyi graphs bi-partitioning 

The most interesting to discuss is the behavior of the algorithm on a given graph and 
the decimation. In particular: Is the function Ai(h, {h 1 ^ 3 }) (|2~lj) continuous in h such 
that any value of the magnetization can be fixed? Does the external field h converge 
in the iterations? Does the decimation achieve low energy states? We choose typical 
Erdos-Renyi random graphs to illustrate the behavior and answer these questions. 

An Erdos-Renyi random graph of average degree smaller that one, a < 1, basically 
looks like a collection of small disconnected trees. Let us hence first discuss how does 
the algorithm behave on a tree. On a tree the belief propagation equations (fTB)) have 
only one possible fixed point for every value of h. For h > all h % ^i = h+di — 1, where 
di is the degree of node i and magnetization m = 1, for h < all h 1 ^^ = h — d% + 1 
and m = —1, and for h = all — and m = 0. 

When fixing magnetization to some value — 1 < m* < 1 the third fixed point is 
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attractive. But provides a wrong value of magnetization m = 0, as in this case the 
function M(h, {h % ~^i}) is a step function. The fixed point also does not provide much 
of a useful information about the cost of splitting a tree on two groups of a given 
size. So in a sense our algorithm does not behave very well on trees, that is a kind of 
unusual situation for belief propagation. The decimation algorithm, however, works 
well and is able to obtain reasonably good partitions even on a tree. This is because 
once a spin is fixed the information propagates and is taken care of correctly. 

But back to Erdos-Renyi random graphs, for mean connectivities above the 
percolation threshold but lower than the satisfiability threshold 1 < a < a s (given by 
([5]), and depicted in Fig.[l} one finds that the algorithm [2] converges to a configuration 
such that on the giant component of the graph all local fields are positive (or negative) . 
Thus all spins on the giant component will be chosen to be +1 (-1). In the rest of the 
graph (that is all the small components) the local fields as well as the external field h 
are negative (positive) but very close to zero. In such a case again, the BP algorithm 
without decimation is not very efficient in actually dividing the components into two 
properly sized groups. However, the decimation algorithm achieves this task quite 
well (note that if one spin on a small component is fixed, than all the other vertices 
orient in the same direction). 



0.8 



0.6 



0.4 



0.2 




1 a t a t , 2 3 
average degree, a 
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Figure 1. Left: The plot shows two phase transitions in partitioning of Erdos- 
Renyi random graphs. The satisfiability threshold a s , eq. JSJ, above which 
the giant component have to be cut to bipartition the graph. And the glass 
transition, a c , at which the belief propagation equations stop to converge and 
replica symmetry breaking is needed to describe correctly the system. Note that 
a s < a c (m = 0) = 1.472 as anticipated in 1221 . Right: Bisection width b on 
Erdos-Renyi graph as a function of the mean connectivity computed by averaging 
over 2 graphs of size N = 100000 with algorithm [2] The data are compared with 
the exact average bisection width b calculated with the extremal optimization 
heuristics for TV = 2000, data from |22| . As the replica symmetric result provides 
a lower bound on the energy and the exact ground states on systems of finite sizes 
are in this case larger that the asymptotic values, the asymptotic value must lie 
between the two curves. The inset zooms into the phase transition region. 

After the satisfiability threshold ([5]), the giant component is bigger than the 
number of vertices that are in the larger of the two groups, so inevitably one will 
have neighbors with opposite spins in the ground state. There are two possibilities: 
(A) BP converges or (B) BP does not converge. If BP does converge, i.e. bellow a c , 
then it converges to a configuration where the giant component is divided into two 
groups (positive and negative local fields) and all the other components of the graph 
are oriented in one direction (the one that has smaller number of vertices on the giant 
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component). In order to fix the proper magnetization on the giant component the 
external field is nonzero even when the total magnetization m* = 0. 

BP does not converge above the replica symmetry breaking threshold a c depicted 
in Fig. [TJ But even in such cases the snapshots of fields are meaningful and 
the decimation algorithm achieves good energies even when the non-convergence is 
ignored, as illustrated in section l4~2l 

In fact on the Erdos-Renyi random graphs there is a first order phase transition 
at zero magnetization. At the transition the derivative of the energy with respect 
to magnetization has a discontinuity. On both sides of the transition a meta-stablc 
state exists with spinodal points at values of magnetization corresponding to the half 
size of the giant component. This phase transition and lines corresponding to the 
meta-stable state and the spinodal point are illustrated in the figure [2 
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0.012 
0.0 1 
* 0.008 
0.006 
0.004 
0.002 


-1 -0.5 0.5 1 

m 

Figure 2. The figure shows the partitioning cost b as a function of m for two 
different Erdos-Renyi graphs with mean connectivities 1.44 and 1.6 (and of sizes 
N = 100000). In the simulation, the messages were randomly initialized for 
m = — 1 and then b was calculated with algorithm [2] Magnetization was then 
slightly increased to ra+Am and messages were initialized with their values 

from simulation with previous m. The dashed curves correspond to the case when 
the system orients the spins on the small components in the less favorable way 
(that is, +1 for m > and vice versa). The dashed curves end at a spinodal point 
where the giant component is divided in half. 

How to understand this phase transition: Consider large positive magnetization, 
in the lowest cost solution the giant component and large part of the small components 
are positive and a small part of the components are negative. As the magnetization is 
decreased the small components are all turning negative, and also parts of the giant 
component turn negative. The external field is negative in that region in order to 
keep the small components negative. Even after half of the spins become negative 
the system does not realize that it is less costly to turn everybody, instead if the 
magnetization is slowly decreased further the belief propagation equations indicate 
that a larger fraction of the giant component should be negative. As the magnetization 
is decreased the negative external field becomes closer to zero, at the point the external 
field flips to positive values the small component turn to positive direction and the 
system realizes this gives much lower cost. This point corresponds to a spinodal 
point. Of course this discussion could be repeated by changing the works positive for 
negative and vice versa. The phase transition, meta-stable state, and spinodal point 
are illustrated in figure O 

If the magnetization is not changed gradually, depending on the initial conditions 
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the algorithm does converge to one or the other of the branches, more likely to the 
lower one. This is a nice property as if more stable divisions are present in real network 
our algorithm might be able to find them (or at least those of them with considerably 
large basin of attraction) . 



4-2. Performance of the BP decimation 

In this section we illustrate accuracy of the decimation BP solver on random 3-regular 
graphs. Regular graphs are in some sense the hardest case for graph bi-partitioning 
as they look locally alike from every point and no apparent structure can be explored 
to decide if two nodes should be in the same group or not. 
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Figure 3. Left: Decimation results for 3-regular random graphs of different sizes, 
compared to presumably exact average ground state energies as computed from 
the extremal optimization heuristics by |14| . Also shown is the asymptotic cost 
b = 0.1138 calculated by 1RSB method. Note that the decimation algorithm is 
far better than the best known algorithmic bound b = 0.16 [19] . Right: The 
plot shows replica symmetric (BP) results, 1RSB results and performance of the 
decimation algorithm for the partition cost b as a function of the magnetization 
m for 3-regular random graphs. The BP population dynamics algorithm was with 
N = 10000, 1RSB solutions were obtained from a simulation with N = 30000. 
The decimation results were were averaged over 10 different graphs, each with 
N = 2000. 

If Fig. [3] we show the average bisection cost achieved by the decimation solvers 
on graphs of different size. We compare to the asymptotic value of the cost and to the 
average values obtained from extremal optimization heuristic of [141 133] that are exact 
(or at least very close to exact), we see that our decimation solver achieves energies 
very closed to the ground states. In particular note that the best provable algorithmic 
bound for 3-regular graph bisection is b = 0.16 [19] which is far above what decimation 
achieves. 

In the right part of Fig. [3] we compare the partition cost as a function of the 
magnetization m as obtained from (a) the population dynamics solving the BP 
equations, (b) resolution of the 1RSB equations from sec. 12.31 under the assumption 
that for every edge the distribution of fields is the same — this being 

called the factorized solution in [29 , and (c) decimation solver run on graphs of size 
N = 2000. 
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4-3. Random regular graphs at zero magnetization 

In this subsection we want to discuss the bisection (zero magnetization) of random 
regular graphs. This case has been treated in [21 [231 HH HS1 I2H] using analogy with 
spin glasses, i.e. Hamiltonian 



with random = ±1 has been solved instead of fixing magnetization to zero via an 
external field. 

Indeed, note that in random regular graph it is more than reasonable to assume 
that the two groups in graph bisection have exactly the same properties and hence the 
first order phase transition that we have seen at m = in the Erdos-Renyi graph is 
expected to disappear. Consequently, the slope of the ground state e(m) at m = is 
expected to be zero, and hence also the value of external field to which our algorithm 
converges is zero h = 0. 

We remind that cavity fields h 1 ^ 3 can be interpreted as a change in the ground 
state energy of ([6]) when link (ij) is removed from the graph. If h is an integer then 
also all h l ~* 3 have to be integers in the the final solution of the problem. The cavity 
equations can then be parameterized by fraction of negative, positive and zero cavity 
fields h i ~ f3 . The only way to achieve zero magnetization is then to set the fraction of 
negative and positive cavity fields equal. And this leads exactly to the same equations 
as Mezard and Parisi obtained in [29 and justifies the approach of J2M231I24, 25, 26ll28j. 
Consequences and generalization of this equivalence will be described in [33] . 

We want to stress that at non-zero magnetization the corresponding external field 
h does not take an integer value and hence no straightforward relation to the spin glass 
problem exists. Also as long as the degree of the graph is not constant there might be 
a room for a first order phase transition at m = due to asymmetries between the two 
groups in the bisection - as illustrated in the Erdos-Renyi graphs. If the first order 
phase transition is present that at m = the external field h ^ and hence again 
no straightforward analogy with the spin glass problem exists. Thus the approach 
developed in this paper is the only one know that is able to treat non-regular graphs 
or non-zero values of the magnetization. 

In Table [1] and Fig. [4] we summarize the known rigorous bounds for bisection 
widths in random regular graphs. We also summarize results of belief propagation 
obtained from our population dynamics, and the results from 1RSB calculation using 
integer values of the cavity fields. Both the latter are only approximation to the full- 
step replica symmetry breaking result that would presumably be exact in this case. 
Finally we compare with performance of our decimation BP solver. In particular 
Fig. H] illustrates how accurate the decimation solver is. Not that the true value of the 
bisection width must lie between the decimation and 1RSB data points. 

5. Discussion 

The main practical contribution of this article is the belief propagation algorithm for 
graph partitioning problem that we anticipate to be useful in the various applications 
of the partitioning problem. We studied the behavior and performance of the algorithm 
on random graphs but we anticipate it will be meaningful also for other families of 
graphs, complex networks in particular. Compared to other partitioning algorithms 
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Table 1. This tabic summarizes the best known lower bound (2nd column, 
[18] [TT], the Bollobas's bound d/4 - yJ{d\n2)/2 is the best known for d > 5) 
and the upper bound (3rd column, [19 1 20. 21_) bounds for random regular graph 
bisection. In the 4th column we give results for the bisection from the population 
dynamics for belief propagation, these numbers are identical to the ones obtained 
in [26^ with non-integer cavity fields. The 5th column gives results of the 1RSB 
calculation with integer fields as developed in |29] , And the final column shown 
performance of our implementation of the BP decimation algorithm for graphs of 
size N = 2000. 
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Figure 4. We plot data from Table [T] rescaled as (2b — ~)/v<f as a function of 
the degree d. According to [2] for large d the true values should converge to the 
ground state energy of the Sherrington-Kirkpatrick model, E = —0.763219. 



BP has the advantage that is provides information about probability with which a 
certain node is in a certain groups. It is also able to see different locally stable divisions 
of the graph - as illustrated by the first order phase transition in Erdos-Renyi graphs 
at zero magnetization. In real world networks the partitioning cost at different values 
of magnetization m may lead to a non-trivial information about communities in the 
network and information about their significance. Note also that our approach is 
straightforwardly generalizable to fc-partitioning the graph into k groups of a fixed 
size. 
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